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ABSTRACT 



The School for New Learning (SNL) was established in 



1972 at DePaul University especially for adult students. SNL students 
are adults, age 24 or older, who want to accelerate their progress to 
a degree by using experience as well as classwork for academic 
credit, and who want to design their own programs with help from an 
advisory committee of experts. The undergraduate curriculum is 
competence-based (that is, focused on learning outcomes), 
student-centered, and interdisciplinary. SNL assesses students on 50 
learning outcomes or competence statements, targeted to and organized 
into 5 domains, each of which is comparable to a discipline found in 
a traditional liberal arts program. An assessment study of this 
program was needed to describe the range of written reports assigned 
in content courses, identify teachers expectations for the written 
reports that they assign, and rate students* performance against 
teacher criteria. The goal of the study was to define problems and to 
propose reforms for improvement. Two theoretical frameworks 
undergirded the study: (1) a psychological model of intellectual 
development in college students ; and (2) an instruct ional design for 
complex problem solving. Methods included analyses of teachers 
descriptions of assignments, categorization of assignments, rating's 
of student papers, and surveys of student perceptions of teachers* 
expectations and criteria. Results, which are various and copious , 
are logged on numerous data sheets. Surveys are appended. (TB) 
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for Adult Students 

Barbara Mirel, School for New Learning, DePaul University, Chicago, Illinois 

AERA 1995 

1 Description of the College (School for New Learning) and its Adult Students 

The School for New Learning (SNL) was established in 1972 especially for adult 
students.SNL students are adults, age 24 or older, who want to accelerate their progress to a 
degree by using experience as well as classwork for academic credit, and who want to design 
their own programs, with help from an advisory committee of experts. SNL enrolls more than 
2^00 students and has 1,800 alumni. SNUs instructional staff includes 24 professors (tenured 
and tenure-track) and 125 visiting faculty. 

The undergraduate curriculum is competence-based (that is, focused on learning 
outcomes), student-centered, and interdisciplinary. Students receive a Bachelor of Arts degree in 
liberal arts studies from DePaul University that is accredited and accepted by graduate schools 
across the country. SNL does not have course requirements for a Bachelor of Arts degree but 
instead assesses students on fifty learning outcomes (competence statements), statements such as 
"Can compare two or more societies with respect to their economic structures or political 
organizations." Competence statements are targeted to and organized into five domains, each of 
which includes ten statements. The following 5 SNL domains are comparable to the disciplines 
found in traditional liberal arts programs: 

(1) Arts of Living (comparable to the humanities and tine arts) 

(2) Human Community (the social sciences) 

(3) Physical World (the physical sciences, math, and technology). 

(4) Lifelong Learning (core/general education requirements, e.g. writing and reasoning) 

(5) World of Work (students' chosen area of concentration) 

Student^ may demonstrate competence either through course work, independent study, or 
through an external assessment of their experiential learning. Eighty-five percent of the liberal 
arts competencies are demonstrated through course work that teachers design specifically for 
particular competencies. Courses are usually offered for two competencies, often for two 
competencies from different domains. The majority of assignments in the liberal arts courses are 
writ*'=^n reports. These assignments are the principle means for demonstrating and assessing 
competence. 

Students may take courses in any order to demonstrate liberal arts competencies in any 
sequence. Therefore, teachers must design each ten week course as a self-contained learning 
experience. They cannot assume that any class of students will enter a course with similar 
background knowledge and experience in that area of study. A ten week class in a quarter is a 
short time for introducing students to a subject and developing their critical capacities . 

Nearly all of SNL's students are full- or part-time employees with many of Chicago's 
leading employers. The main industries employing our students include telecommunications 
and computing (Ameritech, AT&T), banking, finance, and insurance (First Chicago; First' 
Colonial; Chicago Board of Trade; Arthur Anderson); health services (Enterprise, Baxter 
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Travenol); government (Argonne National Laboratory, City of Chicago); and consumer products 
(Kraft, Helene Curtis). 

As with most working adults, many of our students are experiencing reengineering at 
work. A significant proportion of them are managers, team leaders or in career paths that will 
lead to supervisory positions. They look to their liberal arts education to prepare them for the 
changes and the demands of their high perfoimance workplaces. 



2 The Problem and Need for An Assessment Study 

In their dual role as students and employees, SNL learners need a level of intellectual training at 
college that prepares them for the following demands at work: 

• Handling and designing change by focusing on the background assumptions and 
institutional arrangements that shape corporate strategies, communications, and 
interpersonal relations. 

• Understanding the functioning of different business units, multiple ways in which they 
fit together, their interdependencies and outcomes of these relationships. 

• Seeing patterns in new situations, tying them to patterns in other situations, and using 
insights from comparisons to contribute to day-to-day plans and decisions. 

• Using new technologies for enhanced communication and information-sharing, 
retrieving and analyzing data and employing e-mail, groupware, and networks to 
produce and disseminate knowledge. 

• Collaborating in cross- functional teams and becoming well-skilled in conflict 
resolution, negotiation, management of group processes, interest-based problem- 
solving, joint efforts, and self-monitoring. 

• Learning how to learn, learning new roles, and learning to train others. 

These skills are the essential qualities for solid job performance defined by the 
Department of Labor Secretary's Commission on Achieving Necessary Skills (SCANS). For our 
reform purposes, one of the most important aspects of the SCANS report is its call for academic 
institutions to develop curricula to support these skills and competencies. As the SCANS report 
suggests, the liberal arts are a natural "home** for developing the skills and competencies 
proposed by the Commission. For example, systems and technology competencies can be 
addressed in the Physical World ; social systems and information competencies in the Human 
Community. 

Courses in our liberal arts program target a level of intellectual challenge and inquiry that 
falls short of what is demanded intellectually of students in their work environments. Course 
assignments do not provoke the same complexity of thought and facility in approaching 
unstructured problems and situations from many angles that high performance workplaces do. 
Two scenarios illustrate this dissonance between schooling and workplace learning. These 
scenarios, drawn from School for New Learning experiences, typify instruction and assignments 
in undergraduate, liberal arts degree programs for working adults nation-wide. 
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Scenario One 



Maureen directs a ten-person consulting firm that specializes in organizational development. She 
assesses clients' organizational needs and pjroblems and then designs and delivers customized 
training. Typically, clients call her to solve their 'communication problems" in the hopes that she 
will simply deliver some ten-point program for building teams and improving communications. 
Instead Maureen conducts an intensive needs assessment in which, through interviews and 
observations, she fleshes out the issues that lie behind apparent problems (for example, tracing 
problems about team members not sharing workloads to organizational structures that do not 
support required flexible schedules). To interpret needs assessment data, she often shifts between 
interpretive frameworks as newly gathered information suggests alternate focal points. She tests 
her interpretations by creating "what-if scenarios* and trying them out on clients. 

Maureen's job satisfaction comes from the creativity and intellectual challenge it takes to be 
receptive to new information: Putting it together in many ways to get different meanings, relating 
prior cases to present ones, finding the right analogies to explain a case, and looking at findings 
through different "interpretive lenses" to discover significance and priorities. 

In school, she is unchallenged. For example, her "Conflict Resolution" course in the Human 
Community domain assigns an expressive journal and a report asking students to analyze one 
conflict resolution model and give an example of its application. The only other course offered on 
this same subject has similarly introductory assignments. Maureen has no opportimity to tackle 
problems in this subject at a level to which she is accustomed. 



Scenario Two 

Brad is a facilities planning manager in a major telecommunications corporation. Because the 
corporation is in the midst of a year-long reorganization process. Brad continuously handles quick 
turnaround situations that paradoxically require carefully wrought and executed plans . His job 
involves integrating multiple tasks, weighing priorities, managing numerous parallel activities and 
functions at once, adapting to unexpected changes or crises* planning at a level of detail that 
prepares for surprises, and maintaining high team morale in stressful situations. 

For instance, in one case. Brad had no more than two months to open and equip a new office 
staffed with people assxmiing newly created roles. He and his team had to locate and negotiate 
space, to determine infi^tructure requirements, and to supervise installations. In addition, they 
needed to acquire equipment and coordinate with Human Resources to design and arrange training 
for new personnel. When one aspect of the project went awry - the Realtors quit — the rest was 
in jeopardy and required smart decisions on-the-fly. 

In his liberal arts studies. Brad likes literature and the Arts of Living best, learning from these 
courses to see the world in ways he hasn't thought of before. The rich details of fictional worlds 
rest^nate with him, in large part because of the attention he pays to detail in facilities planning. 
Yet he has not experienced literature assignments that ask him to "play" with details in a way that 
challenges him, asking, for instance, why are these details and not others there? What would 
change if certain details were different or if characters interpreted them differently? What details 
are key in distinguishing this situation (theme) from others? 



Our goal is to bridge higher education and work through complex problem-solving. This 
bridge will give students-who-are-workers intensive practice in exploring issues from many 
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points of view in a wide range of subjects and unstructured situations. These academic inquiries 
also will provide opportunities for sustained reflection . 

Until this assessment study, however, the School for New Learning had not examined the 
extent to which assignments in content courses fostered critical thinldng and complex problem - 
solving. Nor had it assessed the extent to which the writing required for the assignments in one 
class build on, spiral with, reinforce or progress beyond the writing required in other classes in 
the same domain. The absence of such a study is common in adult undergraduate programs 
nation-wide. Similarly common is a lack of baseline data on students' writing abilities in content 
courses as gauged against teachers' expectations and criteria. 

3 Research Goals and Methods 

The goals of this assessment study are to describe the range of written reports assigned in 
content courses, to identify teachers' expectations for the written reports that they assign, and to 
rate students' performance against teachers' criteria- The ultimate goal is define problems and to 
propose reforms for improvement 

The design of the study and its methods are as follows: 

• Gather and analyze teachers' descriptions of assignments from all domains. 

• Conduct a pilot study of the Physical World domain. 

• Categorize assignments in the Physical World courses and meet with task forces of 
teachers representing each category of assignment. 

• In each task force, have teachers set their criteria for that type of assignment. 

• Rate student papers gathered from multiple courses against the criteria set by teachers. 

• Survey students for their perceptions of PW teachers* expectations and criteria. 

• Analyze data gathered from all phases of the inquiry to identify problems. 

• Draw conclusions about possible solutions. 

At the time of this presentation, I have completed the assessment of one type of PW 
writing, the intermediate level (the type comprising 2/3 of all PW assignments). For 
intermediate assignments, I have fine-tuned assessment criteria so that raters can rate against 
them; analyzed raters' ratings of 60 sample student papers in the PW intermediate category, and 
evaluated 80 students' survey responses about the criteria that they believe teachers set for PW 
assignments. 

I have developed and proposed a program for improving writing in the content areas in 
the School for New Learning that is still pending. The improvements rely heavily on faculty 
development — workshops for assignment writing and for faculty to collectively coordinate the 
cognitive demands that are implicit in their assignments and integrate and sequence courses and 
assignments within each domain. Improvements also include additional support services for 
students, adding a required upper level writing competence, and creating a faculty position for a 
writing coordinator. 

4 Theoretical Frameworks 

Two theoretical frameworks underlie this study: (1) psychological models of intellectual 
development in college students and (2) instructional design for complex problem-solving. 
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(1) Psychological Models of Intellectual Development. 

Combining the complementary work of Perry and Belenky et al., researchers in cognitive 
development have developed a 4 level model of intellectual development Ln college, with 
learners moving from a lower level of believing that all questions have right answers to a higher 
level of recognizing many legitimate contending positions and a world characterized by 
uncertainty and flux. In this psychological model, the highest level is equivalent to the 
intellectual demands of high performance workplaces (see Table 1). 



Level 1: 


Dualism and received knowledge 


Acquires information provided by authorities 
(unquestioned truths) and learns discrete facts from 
texts without seeing them as interpretations. 


Level 2: 


Multiplicity and sul^ective 
knowledge 


Is receptive to conflicting doctrines and opinions, 
believes everyone is entitled to his or her own opinion, 
and adheres to one's own "right" view on the strength of 
common sense and the legitimacy of one's own "inner 
voice/ 


Level 3: 


Relativism and procedural 
knowledge 


Recognizes that some opinions may be stronger than 
others based on following the "rules'* of inquiry, such as 
providing balanced evidence, citmg sources, looking at 
two sides of an issue. 


Level 4: 


Commitment in relativism and 
constructed knowledge 


Reasons dialectically, explores many points of view at 
once, accepts ambiguity and uncertainty, and links one's 
judgments to context, priorities, prior experience, and 
one's own deliberated stance in the world (for example a 
stance that values change more than continuity) 



Table L Levels of Intellectual Development (Adapted from Krufiss, 1988) 

Two-thirds of our current assignments in the liberal arts target a level of thought and 
inquiry corresponding to levels 2 and 3. For example, an Arts of Living assignment asks 
students to compare and evaluate two novelists' treatments of the human condition. At best, 
students characteristically apply the ''rules" of literary analysis, but they rarely explore the 
multiple meanings of "the human condition" or contextual factors shaping authors* meanings and 
readers* responses. 

In our assessment study, which used criteria established by teachers to rate student 
papers, a large majority of students' papers received low and average ratings in these pro-and 
con- debating assignments. Qearly, many students need continued practice and training at this 
level of inquiry. At present, courses and assignments domain-wide do not systematically provide 
such training. 

In addition, we need to help students who are ready to move beyond this level of thought 
and inquiry. Fewer than one-quarter of all assignments in each domain ask students to conduct 
inquiries or solve problems at level 4. One such assignment involved a student in participant - 
observation in a "theme restaurant," interpreting her observations from many theoretical points of 
view and speculating about a basis for giving them legitimacy. Our reforms strive for more 
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assignments at this level , for more problem-based scenarios and simulations to stimulate such 
higher level thinking, and for more students excelling in them. 

(2) Instructional Design for Complex Problem-solving 

Researchers in instructional design stress that the advanced instruction required for 
complex problem-solving differs in kind not degree from introductory instruction. Introductory 
instruction teaches generic (context-free) and discrete concepts, principles, processes and simple 
relationships between ideas (for example, in the area of risk communication, teaching 
components of communication and factors affecting perception of risk). Advanced instruction, 
by contrast, is richly embedded in contexts, problems, and learners* individuality. It provokes 
students to "dig into** an unstructured area of study to become involved with complex 
relationships between ideas and relevant issues from other disciplines (for example, advanced 
approaches to risk communications ask learners to figure out and test effective ways for a 
national research laboratory to communicate the risks of its research on nuclear energy to the 
diverse population residing in the area ). 

The following findings from this research on complex problem-solving are relevant to 
this assessment study (Schank; Feltovich et al; Brown et al; Cognition and Technology group; 
Goodrum et al.; Perkins and Salomon; Salomon et al.; Cobb): 

1. Goal-based scenarios and simulations that situate learning in authentic contexts ai'e the 
most effective means for inciting advanced learning and for giving students experience 
simultaneously in communication, human relations, and reasoning. Scenarios and simulations 
foster multiple perspectives and flexibility in organizing knowledge in new ways for diverse 
circumstances. 

2. Students need to move past their misconceptions from overly simplified introductory 
learning, such as seeing factors as isolated events rather than as relationships or 
interdependencies. Case studies, goal-based scenarios, and simulations and comparisons between 
them facilitate this process. 

3. Learners need to experience a diversity of inquiries to develop expertise. 
Concomitantly, adults learn best when presented with situations and problems that encourage 
self-directed learning (Lawler, 1991). 

4. Unique assessment methods must be adopted for instruction in complex subjects, 
situations, and problem-solving, including assessmv nt over time; multiple methods for a 
comprehensive enough "montage** of overlapping pictures; evaluations of students' readiness for 
greater complexity; consultation with experts who are actively involved with subject matters and 
problems; broad follow-up surveying; and concept analysis (for example through content 
analysis of journals) to identify continuing sources of difficulty (Feltovich et al.) Our assessment 
methods represent all of these approaches. 

5 Results and Conclusions 

Results are presented in the attached data sheets. I will talk through their significance and 
present the reforms in assignment development and assessment that the School for New Learning 
hopes to pursue based on the results of this study. 
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PW Writing Criteria- Introductory 

Content 

1. Print Sources 

• Use more than one source. 

• Use footnote citations appropriately. 

2. Pointr. of View 

• Must have a thesis/position statement. Develop an ''involved" thesis, not a simplistic one. Its 
not enough to just include a thesis; it must reflect thought - more than a canned position. 

• Develop many sides of an issue but not in a rote way. Give emotional charge to the various 
sides. 

• Analyze the ways in which different points of view relate to each other. 

• In descriptions and observations, compare your perceptions and conceptions to other sources. 
Reinforce your position in the comparisons. 

3 Deveiopment of Ideas 

• Write a clear, concrete, well-focused thesis. 

• Answer the "so what?" question: explain the significance of a perception, experience, 
impression or feeling. 

For example, answer: 

- what does your presentation mean? 

- why are your descriptions or arguments important and why should readers care? 

- what are the relevant complex issues that people have wrangled over for a long time and for 
good reasons (show a recognition of the complexity.) 

• Draw connections between ideas, support for arguments, or descriptions. 

• Have a focus or thesis (your purpose for writing); do NOT just list facts or description like an 
annotated bibliography. 

• Use definitions that provide examples. 
For observations and descriptions 

• Use vivid, exact language to evoke the s^ng or scene, to capture the facts of a situation 
accurately and precisely and to purposefully create an impression or feeling in readers. 

• Use a combination of quotations and your own words to define or describe a phenomenon. 

• For a thesis, you do not need to draw conclusions as you do for arguments. 
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• You do need to draw a conclusion as your thesis when disagreements exist between 
phenomena you observe and descriptions of them in sources . Then you must address and 
resolve the disagreement taking a position that you will support in the paper. 

• For a "non-conclusion" thesis, you may: 

• identify phenomena and their classifications. 

- target the most important things to observe or desaibe based on criteria or traits for choosing 
those things as important. 

• For a bonus and extra advantage, draw broader conclusions from ;you observations or 
desalptions (e.g. in observing flowers, draw larger conclusions about specific issue in 
ecosystems or conservation). 

• To develop ideas, make them finely nuanced and layered (e.g. give details of flower petals, 
neighboring vegetation, etc.) 

• Include your own observations and relate them to source materials. 

• Must fit your reporting into a larger context This context may be; . 

- book versions vs. actually seen 

- family members (compared) 

- ecological interactions 

• Make it clear in your selection of information that you can sort out the important from the 
insignificant. 

• (At times required, at times a bonus) make comparisons that tie together different desaiptions 
or observations. 

Persuasive report 

• Fit your question/conclusion into a larger context (e.g. larger historical view, political context). 

• Convey that your position and issue intellectually and emotionally involve you (by the depth of 
support, the context and significance you bring out. etc.). 

• Develop a seamless interplay between thests-suppcrt-context. 

• Establish at the beginning what people are saying (the conversation.) about the subject 

• Identify a problem that you see in the conversation that you are answering (ue. show that your 
investigation is needed and worthwhile). 

• Identify the argument that you are making and structure your paper around proving it. 

• Do not choose arguments that nobody questions. Focus only on areas of at least some 
disagreement (e.g. Pollution is a problem is not arguable. It is arguable to say The response to 
type of pollutions should be .) 

Selecting and Applying a Model 

• Identify a problem and two or more models. Show how the models shape what you look at in 
the problem. 
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• Develop a thesis that takes a position on the best model to use to account for what is important 
in an issue. 

• Develop support for your thesis ( the superiority of one model over others) through these 
strategies: 

- state the criteria for your comparisons. These criteria must go beyond personal preference or 
desire. Criteria may be multiple, not exclusive. 

- show what the rejected models overlook that is important 

- show ways that rejected models are stereotypic or reductionist and why that isn*t acceptable 

- weigh the comparative strengths and weaknesses of all models; show how in some ways 1 
model is better and in other ways other models are better. 

(Take a finely nuanced view as preferable, not a black & white choice -{black & white choices 
are acceptable but not preferred). 

- support positions with expert testimony or your own interview studies. 

Organization 

Section breakdowns 

• Break sections down appropriately for your writing purpose (inform/observe; prove, explain, 
persuade; explore) 

• If supporting your position by analyzing different sides, organize first by one side then the 
others. 

• Forecast your organizing principle so that readers know where you're headed. 
Ordering and emphasis 

• Create introductions that establish a relevant context and focus on the issues. 

• Draw only conclusions tied to and supported by the ideas you develop in the paper. 
Paragraph level 

• Make sentences relate to each other so ideas flow smoothly. 
Style and Mechanics 

• Vary sentences to create interest and to evoke thoughtfulness and complexity. 
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PW Writing Criteria - Intermediate 



Content 

1. Print Sources 



Ordered by importance: 

1 Use more than 1 timely, credible, and scientific source, 

H = 4+ scientific sources, pubiishedl985 or after OR 2-3 scientific sources after 1989 
M = 2-3 sources, at least 2 of which are scientific, 2 or more published 1985 or after 
L = Only 1 source even if if s timety & aedible OR only 1 sceintific source among many 

2 Identify researchers' credentials, point of view of journal, etc. 

3 Cite direct quotations. Also cite ideas that are new to the students, debatable, or 
figures/findings. 

H = > 1 per page M = 1 per page L = < 1 per page 

(Knock down a level if the citations are inappropriate (e.g. everything's cited) or 

irrelevant (e.g. commonly known information). 

4 Use consistent documentation style. 

NOTE: Cannot get an H if any one of these criteria is an L. 



2. Sources other than print 

• Use personal experience, anecdote: own data gathering; or testimony as a jumping off 
point They must be supported with other evidence. 

• For controversial issues, must use testimony from many experts with different positions. 

• For ail testimony, identify the spea!<er, affiliation, relation to the field (cutting edge? 
accepted?) 

• Demonstrate initiative by designing own study and gathering original data. 



3. Points of View (Thesis) 

• Develop a clear, concise thesis (no neutral, fact-giving or summarizing - e.g. no giving 
all sides to an issue without taking a stand). 

• Maintain the thesis throughout the paper, tie facts to it w/interpretation (not just listing). 

• Show a balance between your POV and others: 

(1) Genuinely engage views contrar/ to one's own, devoting enough detail and space 
to them. 

(2) Must show multiple points of view. Acknowledge, anticipate and respond to 
opposing POV. Use strategies such as: 

- Show the gains & tosses of your own POV 

- Explore basic questions of critical analysis for that topic or subj. matter. 
• Present experience for one POV. 

- Identify opposing positions & why they fall short, including most current challenges. 

- Argue for the need to integrate many views of theories by showing why none alone is 
sufficient. 



Barbara Mirel, Project Director 
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Devetopment of Ideas 

4, Interpreting written sources 

• Identify researchers' credentials: who are they, where and when is the study published 
(situate and identify), validity e.g. sample size. 2 or 3 sentences about the study, 
findings, and conclusions. 

• Draw connections for a reader, telling the relevance of this study to your argument. 



5. Evidence and support for a unifying idea 

No unifying idea = L 

1st priority - Provide sufficient information for the unifying idea 
Bullets have equal weight: 

• Adequately show at a scientific level the link between 'lacts" and your unifying idea. 
(Scientific means showing physical, biological, chemical, geological processes or 
systems) 

H = Details or explains on processes and systems 
M = Names scientific terms for processes or systems 
L = No science - more sociological 

• For topical /controversial issues, must tie causal explanations and reasoning to theory. 
H = Names a theory 

M = Puts the cause and effect in a framework, larger school of thought 
L - No framework 

• Development of support is easy to follow 
H = Clear and easy 

M = Some unrelated, off-base subtopics 

L = Very jumpy and not supporting the unifying idea 

• Give definitions of tenms, concepts, ideas, processes that are relevant to the point 
you're making, not dtetionary. decontextualized definitions. 

H = Defines terms and places them in context, almost always 
M = Sometimes defines and places in context 

L = Doesnl define tenms that need to be defined or uses dictionary defintions 



6. [If Appropriate] Discussing alternative sciences 

• Must acknowledge that a pt>rspective is not mainstream and and WHY it isnl. 

• Demonstrate proficiency in mainstream science in order to discuss or advocate 
alternative sciences. 

• Must tie discussion of "abnonmal" to what*s "normaf (e.g. PW4 & 5). Build on scientific 
rules. laws, first principles - implicitly a explicitly. 



7. [ If Appropriate] Visual support 

• Choose appropriate graphics for one*s purpose, focus, and PW statement. 

• Use graphics for things they show best relevant to your argument (e.g. trends, 
proportions/distributions, comparisons, muttiple points of view, aberrations (normal vs. 
abnomial), muttiple variables. 



Organization 

8. Section breal<downs 



If no thesis = L 

• Use a straightforward organizing principle. e.g.: 
i. State your case 

Start with a statement/conclusion to prove. The more interesting & provocative the 
statement, the better. 

Avoid artificial intro-body-condusion structure. Get right to the point (NOT dictionary 
definitions as an introduction). 

ii. Prove your case. Build on your interests. 

H = Thesis is in the beginning ^ and has a structure for proving 

M = Thesis is somewhere close to the front and there's at least some structure 

L = Thesis isnt there or if s vaguely impled and has no structure for proving 



9. Ordering & emphasis 

1st priority - Order your support/discussion with a logic that matches your thesis or 
positions (not just a breakdown of subject matter that seems objective rather than 
persuasive ). 



2nd priority- Hold readers* interest in leading to your unifying idea- no rule for how much 
or little background comes before the unifying idea - pace of development may be 
longer if you are a stylistically sophisticated writer and can hold reader's interest with 
flow. 

Make it evident that there is a reason for what comes first second, etc. 
H = Thesis and persuasion 

M = Either (a) a thesis but not ordered and emphasized for persuasion (shifts to 
"covering" a lot like a unifying idea OR (b) there's a unifying klea with ordering 
and emphasizing that is appropriate for it. 

L = No thesis or no unifying klea OR has a thesis or unifying Wea but is an "information 
dump." 

TIPS AND TECHNIQUES FOR ACHIEVING 1 OR 2: LITTLE WEIGHTING IN RATING 

• Make opening engaging - be vivid. 

• Provkle headings to show that the paper is heading somewhere. 

• Make all infomiation under a heading a consistent and logical fit with it. 

• Give a title that expresses your unifying klea 



10. Paragraph level organization 



If no paragraphs ^ L 

Transitions 

• Doni mai<e big jumps between paragraphs. 
Unity and coherence in paragraphs 

• Connect the ideas of successive sentences so that there is a flow and readers dont 
need to keep ren'eading to see why an idea is there. 

Record problems as you read, (hoiistically - places that ''stop" you because of 
diconnectedness. etc.): 

H = 1 problem per page M = 2 problems per page L = 3+ problems/page 

NOTE : The whole 1 range = H (e.g. 1 .5. 1 .8). Same for the 2's and 3+'s. 
11. Style and Mechanics 
All bullets have equal weight. 

If there's a tie, the first bullet - internalizing has greater weight than style 
and mechanics. The second priority is concreteness. 

• Show that you have internalized the vocabulary of the issue youYe talking about by 
avoiding jargon or overly specialized terms but still talking in the terms of the field. 

• Use language and details that present the subject concretely. 
H = range of levels and extensive e.g.*s 

M = series of separate instances 
L = not concrete 

• Bring yourself or anecdotal material into the presentation. 
H = stories 

M = mentions self 1 time 
L = No mention of self 

• Avoid sentence fragments and follow the rest of the conventional mechanical and 
grammatical principles for correctness. 

H = great stuff 
M = Passable 
L = Impedes meaning 

• Follow stylistic standards (e.g. effective complex and complex compound sentences, 
sentence variation, figurative language, concise through activQ voice, verbs-not- 
nominalizations, etc.) 

H = Great stuff 
M = Passable 
L = Dysfunctional 



12. Likelihood of Plagarism 

H = Likely plagarized M =Some parts plagarized L - Not plagarized - own voice 



OVERALL RATING 
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PW Writing Criteria-Advanced 

Content 

1. Print Sources 

• Use newspapers, popular journals etc. to find/discuss a topic and its importance. 

♦ Use sources that give in-depth scientific explanations to analyze the question(s) related to the 
topic. 

♦ If relevant, use books and other sources cited on the teacher's bibliography. 

♦ No need to evaluate other researchers* studies on the basis of specialized design and 
methodology. 

(Students should look at such issues as speculating beyond the data or the representativeness 
of a sample but not at more specialized validity issues) 

• Read widely, discuss hypotheses that others* have advanced, identify faults with these 

hypotheses that show the need for your own. 

i» 

• Should have at least 3 sources (but 20 handled badly is not necessarily good!). As a rule of 
thumb- the # of sources should be proportionate to the length of the paper. 



2. Sources other than print 

• Use personal experience or testimony as a jumping off point. They must be supported with 
other evidence. Testimony must be supported by proving the credibility of the speaker and his 
or her qualifications as an expert. 

• Anecdotal evidence by itself is not good enough support or proof for a point. It is a good way.of 
showing why you chose a subject. 



3. Role of Perspectives 

♦ Must develop a question or hypothesis as the central focus of your report. The framing of the 
question is essential. 

« Must develop an explanatory question. 

♦ Develop a question by synthesizing and interrallzing from other sources. You should not simply 
be take one side in an "opinion-debate" nor ask the same question(hypothesis) that youVe 
already read. 

♦ Identify the sources your question came from and their credibflity. 

♦ Should ground your inquiry in a theory about the topic youVe discussing. Before interpreting 
data, must state the underlying scientific prindple behind the issue (e.g. global wanning). 

« Must draw a clear relationship between your data and conclusions. 

♦ Show how your conclusions drawn from the data are affected by underlying paradigms, 
philosophies, theories . 

- Depending on the topic, bring in contending theories (e.g. PW2). 
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- Get across that there are mulitple conclusions and ways of looking at an issue depending on 
one's philosophical perspective, (e.g. You can have contending theories that are based cn the 
same epistemology or different epistemologies and different theories) 

- It's extra (not required) if you critique contrasting perspectives and tie them in with the internal 
logic of your own perspectives 

• Demonstrate an understanding that science is iterative and you never reach final proof. 



4. Developm ent of Ideas 

• Provide enough information and paraphrasing to lay a groundwork for your ideas. But 
groundwori< information isnt adequate support on its own. You need to organize it and tie it to 
your question. 

• Overall, demonstrate an understanding of scientific principles and the ability to teach them, 

- This does not imply that discussions must be about controversial topics. But it does imply 
assimilating knowledge about scientific phenomena and relating it to other things. 

. Presenting the underlying science is essential - i.e. the wori<ings of the phenomena and how 
they lead to effects. 

• Develop your presentation by bringing your own insights into the discussion even if you are 
allied with someone else's views. 

• Must select the most important facts to communicate to others. Effective selectivity rests on: 

- Tying selected information to your question, purpose, or conclusion (not extraneous, i.e. it 
must lead to a point). 

. Selecting information that justifies what you are arguing and where you are heading. 

• Develop ideas in a way that shows you know where l}ringing in opinions is and is not 
appropriate. Give opinions in only appropriate sections (see below. Breakdowns). 

• If you make any conclusions you must provide evidence. Conclusions warranting evidence 
include your main point but not your "set-up" or ideas and opinions that are generally accepted. 

- Reference to someone else's study is adequate evidence. 

- In giving and developing evidence, explore all sides adequately. Show contending alternative 
perspectives. When it is difficult to find numerous alternatives, focus on analyzing differences, 
e.g.: 

Compare historical periods and show reasonable evidence for differences being tied to 
different situations. 

Show differences in the costs and benefits of doing one thing or another. 

• Must define concepts or temis when they have many uses, novel uses, or are so overused as 
to be meaningless(e.g. environment, hazardous waste). Explicated i.e. say. "here's how I'm 
using the term and what it means." 

- Do not use dictionary definitions. 
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• Must tie defintions to the way the scientific community uses the term or concept. If you can*t 
get a definition from source article (e.g. co-enzyme), go to textbooks. 

Using visual evidence 

• In developing ideas visually, must accompany the visuals with text that interprets them for the 
point you are making. Must make the significance to your visual point dear and explicit. 

• Reports should include visuals to indicate that you know what information is best expressed 
verbally and visually in science. 

Organization 

1 . Breakdowns of infomiation 

• Separate information and include opinion/argiment as follows: 

- Set up a context- a little argument. 

- Introduce question - no arguing. 

- Relate possible hypotheses: yours and others, no argument. 

- Discuss why your question is worthwhile - may argue. 

- Review the literature - be objective; end by saying how it all relates to your questions - not 
evaluating the studies (i.e. no arguing) 

•Evaluate studies (may argue) 

• Methods (optional)- say what evidence was gathered, how subjective (anecdotal evidence) 
was used - no arguing 

- Results of study(ies) - no opinion/arguing. 

- Conclude - may argue. 

• Proportioning of sections depends on the topic. 

• Organize according to where a topic leads, not according to the competence statement. 

• Headings are preferred by not mandatory. A title for the paper is not important. 



tyle and mechanics 
Use standard writing style and mechanics. 

Rrst person is all right, but not in certain sections (Ike the literature review). 



TEACHERS* DISTRIBUTIONS 



Content 80 70 #1 

Organ. 20 20 #1 

Style/ Mechanics 0 10 #3 



ERIC 



10 



I 



Survey on Writing in the P\V noinain 

The School for New Learning is conducting this survey as part of a study aimed at 
improving writing instruction in all domains. This survey focuses only on writing in 
courses offered tor PW competences and should take about 20-30 minutes to complete. 
Your responses will help us to identify important patterns in PW assignments and the 
expectations that you and your teachers ha\ e had for these assignments. We very much 
appreciate your help and cooperation. 

Name Phone 

LWhat was your educational background before coming to SNL? 

HS i # yrs. Community college # yrs. 4 yr. college Other: 

2. What year did you start SNL? 19 

3. What is your concentration in the World of Work domain? 

4. Which PW comp)etences have you fulfilled? (circle all that are relevant) 

PWl PW2 PW3 PW4 PW5 
PWA PWB PWC PWD PWE PWF 
PW9 PWIO 

5. Have you taken the College Writing Course? (check one) Yes No 

If yes, when (what year)? 19 

6. Circle which, if any, PW courses and competences that youVe taken or plan to take are: 

• Relevant to your educational goals? Course names: 

PWl PW2 PW3 PW4 PW5 
PWA PWB PWC PWD PWE PWF 
PW9 PWIO 

• Relevant to your future professional goals? Course names: '. 

PWl PW2 PW3 PW4 PW5 . 

PWA PWB PWC PWD PWE PWF 

PW9 PWIO 

• Relevant to your current job? Course names: 

PWl PW2 PW3 PW4 PW5 

PWA PWB PWC PWD PWE PWF 

PW9 PWIO 

• Relevant to you personally? Course names: . 

PWl PW2 PW3 PW4 PW5 

PWA PWB PWC PWD PWE PWF 

PW9 PWIO 

7. How much time/week do you usually spend studying or preparing for a class? hrs. 

ABOUT PAPERS 

8. How much time do you spend writing a term paper? ^hrs. 
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9. When an instructor gives vou a paper assignments how often do you ask questions if 
the requirements are unclear? (check one) Always Sometimes — Never 

10. How often do you talk to other students about the assignment to make sure that you 
have it clear? (check one) Always Sometimes Never 

1 1 . What would you consider to be some unrealistic criteria for a paper (for example ho 
long is too long? how many citations are too many to expect? What else?) 



12. How often do vou feel you know the grade your paper is apt to receive before turning 
it in? (Check one) ' Always Sometimes Never 

13. Name resources that you have used for writing a scientific paper: 

Journals • • 

Books^ - 

Other.^^ 

14. What qualities make a resource credible? . 



15. Can you think of any scientific or social theories that ypuVe talked about in PW 
courses? What are some of them? 

16. Think over some of the papers that youVe written for PW competences. Think 
example of a paper you wrote Uiat discussed some theory. 

What competence did you write for? 

What was the main point? 

What was your purpose in bringing in theory? 

What aspects of theory did you discuss and why? 

What else besides theory did you discuss? 



17. How did you organize your paper so that readers knew where you are headed? 



3 

18. What do you think your teacher was looking for in that paper? For each possible 
expectation listed below, circle the number ( L 2, or 3) that indicates its importance to your 
teacher. / is very important 

2 is somewhat important 

3 is unimportant. 

12 3 1 To cite many different objective facts. 

1 2 3 2. To present and explain 2 positions on an issue. 

1 2 3 3. To present and explain more than 2 positions. 

1 2 3 4, To explicitly say why the facts you discus^, are important for your reader. 

1 2 J 5. To include quotations from sources. 

12 3 6. To give information that proves that sources or people quoted are credible, 

1 2 3 7. To use appropriate citation form and style. 

12 3 8. To explain in detail how one cause results in an effect. 

12 3 9. To explain many causes, their relationships, and how they result in effects. 

12 3 10. To analyze how causes change due to context and circumstance. 

12 3 11. To explain the "big picture" implications of cause and effect relationships. 

12 3 12, To evaluate why different points of view are strong, weak, or both. 

12 3 13. To overtly identify the main principles, rules or ideas that you base your 
judgments on, 

12 3 14, To identify weaknesses in your own position even though it's a valid position. 

12 3 15, To discuss your own experiences as evidence. 

12 3 16, To explicitly explain how your experiences support or contradict other 

evidence you present. 

12 3 17, To conduct your own surveys or experiments about an issue. 

12 3 18, To show that you can take a complex approach to a complex subject 

12 3 19, To write at least three pages. 

20. Other? 

19. In which, if any, of the above areas did you disagree with your teacher*s emphasis? 
(Circle one or more numbers of the above expectations.) 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 

20. Have most of your PW teachers put importance on the same things? (check one) 

Always Sometimes Never 

20.1. [If sometimes or never] What are some important differences? In what 
ways, if any, are differences tied to courses, teachers, competence statements? 



21. When you have written about a controversial scientific issue or theory (like aging 
theory or homeopathic medicine), have you approached it differently from noncontroversial 
issues? If so. what have you done differently? 



22. How much should teachers be influenced by writing mechanics and style in grading a 
PW term paper? (Check one) , 

Lower it by 1/2 a grade if bad Lower it a whole grade Eton t lower 

Reward good style and mechanics even if the content isn't ftilly developed 

23. When using an expert's testimony to prove a point, what additional information, if 
any, do you need to provide in order to make this testimony convincing? 



24. Do have any additional comments about teacher's expectations for PW papers? 



Thank you.We hope to write up our findings in a report or article. Would you like 
us to send you a copy when it is finished? Y N 



Name: 
Address: 
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Student SurvQr Responses 
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Very 

Important 

Gte many objective facts 39% 

Present and explain 2 

positions on an issue 40% 

Present and explain more 

than 2 positions 14% 

Say why the facts you 

discuss are important to 

readers 58% 

Include quotations from 

sources 37% 

To prove that sources or 

people cited are credible 38% 

Use of appropriate citation 

form and style 33% 

To explain in detail how one 
cause results in an effect 50% 

To explain many causes & 

their relationships & effects 29% 

To analyze changes in 

causes based on conditions 3 1 % 

To explain the bigger picture 
affected by causes & effects 60% 

To evaluate why pts. of view 

are strong, weak or both 25% 

To identify principles, nxles 

or ideas behind judgments 55% 

To identify weaknesses in 

own position even if valid 14% 

To discuss own experiences 

as evidence 27% 

To explain how experiences 
contradict or support 25% 
evidence 

To conduct you own 

surveys or experiments 17% 

To take a complex approach 

to a complex subject 29% 

To write at least 3 pages 39% 



Somewhat Missing 
Important Unimportant Data 



52% 

37% 
35% 

36% 

39% 

37% 

23% 

36% 

42% 

46% 

25% 

38% 

33% 

42% 



44% 

37% 

42% 
21% 

24 



9% 
23% 

52% 

6% 

24% 

25% 
19% 

11% 

27% 

19% 

13% 

35% 



42% 
33% 
29% 

44% 

21% 
31% 



4% 

3% 
2% 



2% 
2% 



2% 
2% 
2% 

2% 

8% 
9% 



Preliminary Findings: Student Surveys & Student Essays for PW Assignments 



1 Student Surveys 

Percentage of students rating the following teacher expectations as not very important: 

87% To present and explain more than 2 positions on an issue. 

85% To identify weaknesses in your own position even though it*s a valid position. 

81% To conduct your own surveys or experiments about an issue. 

73% To evaluate why different points of view are strong, weak; or both. 

73% To explicitly explain how your experiences support or contradict the other evidence you 
present 

71% To discuss your own experience as evidence, 

69% To explain many causes and their relationships and how they result in an effect 

64% To show that you can take a complex approach to a complex subject\ 

2 Ratings of Student Papers 

42 papers rated against teachers' criteria for PW assignments asking for cause-effect and pro- and 
con arguments. 

Overall ratings: 

62% Low 
12% . Average 

24% Mixed low and average ratings 
2% Mixed average and high 
0% ffigh 

(74% reliability) 

Ratings by Criteria: 

Use of print sources (69% reliability) 

62% Low 2% Average 5% High 24% Mixed low and average 4% other mix 

Use of sources other than print (personal experience, anecdote, own data gathering, 
testimony,)(95% reliability) 

93% Low 2% Average 2% Mixed low and average 2% mixed high and low 

Presentation and development of a clear and concise point of view (thesis) (88% reliability) 
83% Low 2% Average 10% Mixed low and average 5% Mixed high and average 

Justificationofthecredibility and relevance of written sources (98% reliablity) 
98% Low 2% Mixed low and average 

Other: 

90% reliability Organized for meaning and purpose - 86% Low 

60% reliability: Style and mechanics - 45% average 33% Mixed average and low 



PW Intermediate - Ratings Results 



Reliability \ 



Low Av. HI] 



L/A A/H H/L 
24% 5% 2l~ 



TOitsoiffces 2^ 
Sources other ttian print 93% 2% 
Point of View (Thesis) 83 % 2% 



Interpreting written 
scuices 

Evidence and support 

Section Breakdowns 

Ordering and Emphasis 

Paragraph Level 
Organization 

Style and Mechanics 

Likelihood of Plagarism 



'0 - 



LOW 

86% 
46% 
Low 

13% 
48% 



reliab 

ility 

5% 

19% 2% 

reliab 
ility 

45% - 
12% - 



2% - 2% 

10% 5% - 

2% - 

7% 2% - 

29% 2% 2% 

33% 7% 2% 

33% 2% 5% 



95% 
85% 
98% 

35% 
90% 
66% 
33% 

57% 
57% 



OVERALL PAPER 



61% 12% - 



24% - 



74% 
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Preliminary Findings: Student Surveys & Student Essays for PW Assignments 



1 Student Surveys 

Percentage of students rating the following teacher expectations as not very important: 

87% To present and explain more than 2 positions on an issue. 

85% To identify weaknesses in your own position even though it's a valid position. 

81% To conduct your own surveys or experiments about an issue. 

73% To evaluate why different points of view are strong, weak, or both. 

73% To explicitly explain how your experiences support or contradict the other evidence you 
present 

71 % To discuss your own experience as evidence. 

69% To explain many causes and their relationships and how they result in an effect 

64% To show that you can take a complex approach to a complex subject\ 

2 Ratings of Student Papers 

42 papers rated against teachers* criteria for PW assignments asking for cause-effect and pro- and 
con arguments. 

Overall ratings: 

62% Low 
12% Average 

24% Mixed low and average ratings 
2% Mixed average and high 
0% High 

(74% reUability) 

Ratings by Criteria: 

Use of print sources (69% reliability) 

62% Low 2% Average 5% High 24% Mixed low and average 4% other mix 

Use of sources other than print (personal experience, anecdote, own data gathering, 
testimony,)(9S% reliability) 

93% Low 2% Average 2% Mixed low and average 2% mixed high and low 

Presentation and development of a clear and concise point of view (thesis) (88% reliability) 
83% Low 2% Average 10% Mixed low and average 5% Mixed high and average 

Justification of the credibility and relevance of written sources (98% reliablity) 
98% Low 2% Mixed low and average 

Other: 

90% reliability Organized for meaning and purpose — 86% Low 

60% reliabihty: Style and mechanics — 45% average 33% Mixed average and low 
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